Papers with runtime speed
Text Compression for Efficient Language Generation (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing models rely on sub-word tokens for text generation, but there is no evidence for a more efficient way to generate text. |
| Approach: | They propose a hierarchical transformer language model capable of text generation by compressing text into sentence embeddings and employing a sentence attention mechanism. |
| Outcome: | The proposed model achieves an up to an order of magnitude improvement in FLOPs efficiency and a threefold increase in runtime speed compared to equally-sized models in the low-size regime. |
SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have remarkable emergent abilities across various tasks, yet their performance on complex reasoning and planning tasks remains suboptimal. |
| Approach: | They propose a tree-search-based reasoning framework that encourages the exploration of intermediate steps and a round-scheduled strategy to manage draft model dispatching. |
| Outcome: | The proposed framework improves runtime speed and GPU memory management concurrently and handles multiple iterations for thought generation and state evaluation. |